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1,  Introduction 
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In  this  paper  we  consider  a hazardous  inspection  model.  We  are 
given  a device  which  operates  throughout  a number  of  periods  and  which 
in  each  period  is  subject  to  failure.  Failure  of  the  device  is  directly 
observable.  Prior  to  failure  the  device  will  enter  a state  in  which  it 
is  functioning^  but  in  an  impaired  manner.  This  state  can  be  detected 
only  by  performing  an  inspection.  Once  the  machine  is  known  to  be  in  the 
impaired  state^  appropriate  action  may  be  taken  to  prolong  the  remaining 
life  of  the  device.  In  this  respect  inspection  is  valuable.  However^ 
the  act  of  inspecting  the  device  when  it  is  not  impaired  may  itself 
cause  the  device  to  become  impaired.  In  this  respect  inspection  is 
hazardous.  This  paper  deals  with  the  determination  of  inspection  policies 
which  maximize  the  expected  lifetime  of  the  device. 

The  above  model  is  very  appropriate  for  situations  where  the  cost 
of  inspection  is  negligable  relative  to  the  cost  of  failure.  One  important 
example  is  the  use  of  X-rays  to  detect  cancer.  Currently  there  is  con- 
siderable controversy  as  to  whether  X-rays  may  themselves  cause  cancer  and^ 
granting  at  least  their  potential  for  causing  cancer^  whether  or  not 
they  should  be  employed.  (See  [3]  for  a detailed  list  of  references  on 
this  subject.)  Another  prime  example  is  the  inspection  of  nuclear  reactors. 
Since  the  largest  single  cause  of  malfunctions  is  human  error  [ll],  a 
fundamental  question  is  do  human  inspections  create  more  problems  than 
they  solve. 

A number  of  authors  have  studied  inspection  models  [1]^  [5]^  [6], 

[7],  18].  These  authors  have  all  assumed  a cost  structure  for  inspection 
and  repair^  and  have  sought  to  derive  inspection  and  repair  policies  which 
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minimize  the  overall  cost  of  operation.  In  none  of  these  papers  was 
inspection  assumed  to  have  any  effect  (good  or  bad)  on  the  true  state  of 
the  device.  Wattanapanom  and  Shaw  [lOj  consider  a hazardous  inspection 
model  in  which  the  device  has  an  exponential  failure  distribution  in  the 
absence  of  any  inspection.  Each  inspection  either  causes  insnediate  failure 
or  else  increases  the  failure  rate.  They  derive  algorithms  for  finding 
inspection  policies  which  minimize  the  overall  operating  cost. 


2.  The  Model 

Consider  a device  whose  operation  can  be  classified  into  one  of 
three  categories:  fully  functionalj  functional^  but  impaired;  and  failed. 

The  failed  state  is  directly  observable  but  one  can  distinguish  the  partially 
functional  state  from  the  fully  functional  state  only  by  performing  an 
inspection.  Inspection  is  perfect  (i.e.,  the  true  state  is  always  revealed) 
and  instantaneous. 

The  inspection  model  will  be  formulated  as  a Markov  decision  process 

[1+].  We  let  X denote  the  true  state  of  the  device  at  the  start  of 
n 

period  n (before  inspection) , where  the  possible  states  are 


True  State 


Description 


l,rr  or 

Section  ^ 

\ 3:fl  Section  □ 

I ^ D 
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fully  functional  (OK) 
undetected  partial  failure  (UP) 
detected  partial  failure  (DP) 
failed  (F)  . 
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Let  a denote  the  action  taken  at  time  n where  the  possible  actions 
n ’ 


are 


Action 


0 

1 


Description 
do  not  inspect 
inspect  • 


We  make  the  usual  Markov  assumption  that  the  future  is  in  some  sense 
independent  of  the  past.  Specifically  we  assume  that 


Pr{X^^l  = 2^  •••,  f*) 

■ ■ J'*.-  ’J  • 


j—  4. 


Define 


Qlj(»)  - . J|X__  . 1,  , . . 0,  1;  J , 1,...,!, 


We  take  the  one-step  transition  matrices  Q(0)  and  Q( 1)  to  be  as  follows. 


Q(o) 

(do  not  inspect) 


Q(i) 

( inspect) 


(OK) 

(UP) 

(DP) 

(F) 

(OK) 

(UP) 

(DP) 

(F) 

(OK) 

l-a 

0 

o. 

0 

0 

0 

(OK) 

— 

“l 

0 

0 

(UP) 

0 

1-p 

0 

P 

(UP) 

0 

0 

1-3 

3 

(DP) 

0 

0 

1-r 

r 

(DP) 

0 

0 

1-r 

r 

(F) 

0 

0 

0 

1 

(F) 

0 

0 

0 

1 

1 


I "fiiKTir*  1 
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We  assume  that  0 < a^,  3,  r < 1 and  that  and  P > r 

( see  below) , 

Note  the  following  features  about  the  model. 

1.  Once  a device  enters  a state^  it  never  returns  to  a lower-numbered 
state. 

2.  Once  a device  leaves  state  1^  the  transition  probabilities  are  the 
same  regardless  of  whether  or  not  it  is  inspected. 

5.  Once  the  device  enters  state  3 or  4,  the  decision  process  is  completed^ 
because  the  future  evolution  of  the  device  is  independent  of  the  actions 
taken.  (This  is  also  true  once  the  device  enters  state  2;  however^ 
state  2 is  indistinguishable  from  state  1 to  an  observer.) 

U.  The  assumption  that  0!^^  > 0!^  means  that  inspection  is  more  hazardous 
to  a fully  functional  device  than  is  no  inspection. 

5.  The  assumption  that  P > y means  that  once  partial  failure  has  been 
detected^  measures  can  be  taken  which  reduce  the  periodic  probability 
of  failure  of  the  device. 

The  information  available  to  an  observer  is  whether  or  not  the 
system  is  in  one  of  the  states  { 1^  2)^  or  state  3,  or  state  U,  and  if 
in  state  1 or  2^  how  many  periods  have  elapsed  since  the  last  inspection. 
[Note  that  the  last  inspection  must  necessarily  have  indicated  the  device 
to  be  OK.]  Thus  we  define  the  following  observed  states. 


Observed  State 


Description 


device  failed 


j > 1 


detected  partial  failure 

device  not  failed;  last  inspec- 
tion OK;  last  inspection  j 
periods  ago 


The  observed  state  transitions  also  have  the  Markov  property.  We  let 
denote  the  observed  state  of  the  device  at  period  n and 


= j|z^  = i,  a^  = a}  , a = 0,  1;  i,j  > -1 


Also  let 


K.  = Pr{X  . = 2|X  = 1.  a = 1.  a , = 0.  0 < k < 1}  . 
J n+j  n ^ n ’ n+k  ' * 


L.  = Pr{X  . = l|x  = 1,  a = 1,  a , = 0,  0 < k < j)  , j = 1,  2,  , 
j n+j  n > n ' n+k  ' j j , j t > • 


N.  = K.  + L.  . 

J J J 

Note  that  probability  that  the  true  state  is  2 (UP)  given 

the  observed  state  is  j and  probability  that  the  true 

state  is  1 (OK)  given  the  observed  state  is  j. 
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Proposition  1. 

A.  do  not  inspect 


i) 


1,  j = -1 

otherwise 


r,  j = -1 


,0,  otherwise 


j = -1 


iii)  P.^j(O)  =aL.+K^(l-p))/N^  , j = i 


= i+1,  i > 1 


otherwise. 


B,  inspect 


i) 


io. 


1-  j = -1 


otherwise 


ii) 


r,  j 

Po^jd)  =<;i-r,  j = o 


otherwise 


iii) 


Ki(l-P)/Ni, 

Li/Ni, 


j = -1 
j = 0 

j = 1 

otherwise. 


i > 1 
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Proof.  Parts  A.i-ii  and  B.i-ii  are  obvious; 

P.  ,(0)  = Pr(X  , = U,  X = 2|Z  = i} 

i^-1  ' n+1  ^ n n 

= Pr{X  , = 4|X  = 2).Pr{X  = 2|z  = i)  = ^K./N.  . 
n+1  n n n 3 3 

Also^  it  is  clear  that  = 0 all  j -1,  1+1,  so  that 

Pf  ^(0)  + P^  l^ls  proves  part  A.iii.  To  prove  part  B.iii  note 

that 

P.  ,(1)  = Pr{X  , = U,  X = 2|Z  = i] 

1^-1'  ^ n+1  » n n 

= Pr{X  , = i4-!x  = 2}.Pr{X  = 2|z  = i)  = ^K./N.  . 

n+1  n n n i i 

Also 


i-2  / 1-a  V k 

i)  K = a(l-ti)''  +a(l-a)(l-p)  ^ E (jr^)  , i>l 


ii)  K.  = a^(i-P)"'^  4-  Vl-ap  [(l-P)""'  ' > 

i > 1,  % ^ 0 


i-1 


i-ln 


iii)  K.  = + (i-1)  aQ(l-a^)(i-a^)"'^  , 


i > 1,  = 3 


[Note:  By  convention,  the  summation  in  B.i  is  zero  when  i - 2 < 0. ] 


Proof. 

h • ‘K0<^<  llx„  = 1,  ^ . I,  . 0,  0 < k < 1) 

1-1 


(l-ap(l-Q:j,) 


>•  '‘l  • ' 2,  ° < I'  < ‘l\  * ^ V - \.k  . 0,  0 < k < 1) 

Z Pr(X„^,  . 1,  0 < k < «,  . a, 


£=2 


n+j 


« < J < 1|\-  I,  p„-  1,  Vk-  “• 


.a,(l.^)‘'‘^.  Z (1-0  )(l.a  • a (l.s)^''' 

^ f=2 


i.2  /1-aQik 

= 1-P)  + cCqC  l'*^]^)  ( I'P)  ( i_p  / 


This  proves  ( i)  from  which  (iii)  follows  directly.  To  prove  (ii),  note  that 


<cd«7-i'ar.JE‘_ 


f 


p • 


K. 

1 


1 - 

1 - (i-^/Ci-P) 


Multiplying  the  numerator  and  denominator  of  the  second  term  by  ( 1-3) 
yields 

1.1 

_a^(i-3)  +aQ(l-ap  ^ 

from  which  ( ii)  follows  directly.  □ 


To  allow  for  the  possibility  of  alternate  failure  modes^  competing 
risks  and  the  like^  we  will  assume  that  every  period  there  is  a probability 
of  secondary  failure  1-6.  We  allow  the  possibility  that  6=1  in  order 
to  cancel  this  feature  of  the  model. 

It  turns  out  that  the  parameter  6 acts  almost  exactly  like  an 
ordinary  discount  factor.  Clearly^  this  aspect  of  the  model  could  have 
been  incorporated  into  the  transition  matrices  Q( • ) directly.  However^ 
our  treatment  shows  the  discount- factor- like  nature  of  6 and  keeps  the 
transition  matrices  Q(’)  more  simple. 

We  now  consider  the  main  objective  of  this  paper,  the  determination 
of  an  inspection  policy  which  yields  the  maximal  expected  life  of  the  device. 
Let  V(s,n)  denote  the  maximal  expected  remaining  time  until  failure  of 
the.  device  given  the  device  currently  is  in  (observed)  state  s and  the 
device  is  to  be  destroyed  n periods  in  the  future  (if  it  survives  that 
long).  Let  V( s)  denote  the  maximal  expected  remaining  time  until  failure 
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of  the  device  given  it  is  currently  in  (observed)  state  s.  As  is  rather 
generally  true  of  dynamic  programs^  the  finite  horizon  optimal  values 
V(s,n)  converge  to  the  infinite  horizon  optimal  values  V( s)  as  n 
approaches  infinity  [9] . That  is 

V(  s)  = lim  V(s,n)  ^ s = -1,  0,  1,  ...  (1) 

..  n 00 

The  recursive  relations  among  the  Vs  are  given  below. 

Proposition  3. 

A.  Finite  Horizon 

i)  V(s,0)  = 0 s=  -1,0,1, 

ii)  V(-l,n)  = 0;  V(0,n)  = (l-(6(l-r))'')/(l-6(l-r))  , n = 1,  2,  . 

iii)  V(s,n)  = 1 + 6 max{Kg(l-p)  V(0,n-1)/N^  + V(l,n-1)/N^, 

(K^(l-6)  + L^)  V(s+l,n-l)/N^}  , s,n  > 1 

B,  Infinite  Horizon 

i)  V(-1)  = 0,  vco)  = i/(i-6(i-r)), 

ii)  V(s)  = 1 + 6 max{K  ( 1-P)  V(0)/n  + L V(  1)/N  , 

s s s s ^ 

(K^(l-p)  + Lg)  V(s+1)/N^),  S>1. 


Proof. 


'I 


V(0,n)  = Y.  Pr( remaining  life  of  device  exceeds  k.| device 
k=0 

currently  in  true  state 

k 

= z mi-r)r 

k=0 

= (1  - (6(i-r) )")/(!-  B(i-r))  . 

The  formula  for  V(0)  follows  directly  from  the  above  by  taking  the  limit 
as  n approaches  infinity. 

By  the  so-called  principle  of  optimality  of  dynamic  programming  [2]^ 

00 

V(s,n)  = max  {1  + & Z ^(j,  n- 1) ) . (2^ 

a = 0,1  j=-l 

Recalling  Proposition  1,  the  recursive  formula  for  V(s,n)  follows 
immediately.  The  recursive  formula  for  V(  s)  follows  from  taking  limits 
in  the  above  [9].  C; 


3.  Preliminary  Analysis 

We  begin  the  analysis  of  the  model  by  investigating  the  behavior 

of  the  quantities  K./N.  and  L./N.. 

11  11 
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Proposition  4. 

i)  If  Gc^  > (^0,  then  is  strictly  increasing  in  1,  and 

L./n.  and  + L^/N^  are  strictly  decreasing  in  i. 

ii)  If  Q!q  = then  K./N.,  L^/N^,  and  (K.(l-p)  + are 

constant  in  i. 

iii)  If  then  is  strictly  decreasing  in  i^  and 

^i^^i  strictly  increasing  in  i. 


Proof. 


K.  L - K.  , 

1 1-1  1-1  i 


v-i  - (k.  :.l.  ,) 

1 i'  ' 1-1  i-l' 


so  is  increasing^  constant,  or  decreasing  according  to  whether 

^ is  positive,  zero,  or  negative, 


K.  L.  , - K.  , L. 
1 1-1  1-1  1 


i-2  ,l-a,k 


- (i-%) 


- «i(i-“o)  - V'-“i)  (tit)  } 

= (l-a^)(i-aQ)^-2  {Qo-aiP}  . 
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This  proves  the  assertions  regarding  The  other  assertions  follow 

directly  from  the  formulas 

L^/N^  = 1 - K^/N.  , 

(K^(l-p)  + L^)/N^  = 1 - pK./N.  . □ 

Using  the  above  proposition,  we  can  now  compare  V(s,n)  and  V( s) 
for  various  values  of  s. 

Theorem  1. 

1)  If  ^ then  V(s,n)  and  V(  s)  are  non- increasing  in  s > 1. 

ii)  If  *^Q  ^ then  V(s,n)  and  V( s)  are  non- decreasing  in  s > 1, 

Proof.  ( i)  For  n = 1,  V(s,n)  is  constant  for  s > 0 and  therefore 
non- increasing.  Now  using  induction  assume  that  V( s,  n-1)  is  non- increasing 
in  s > 1.  The  two  arguments  of  the  maximization  operator  in  the  recursive 
expression  for  V(s,n)  (Proposition  3)  are 

Kg(l-P)  V(0,  n.l)/N^  + V(l,  n-l)/N^ 

= (1-P)  V(0,  n-1)  + Lg(V(l,  n-1)  - ( 1-P)  V(0,n- 1)  )/N^,  (5) 

and 

(Kg(l-P)  + L^)  V(s+1,  n-l)/N^  . (4) 


\ 
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k 

I 

I 

! 

! 

!, 

1 

[ 


I 


By  Proposition  4 (K  ( 1-P)  + L )/n  is  decreasing  in  s and  by  hypothesis 

s s s 

V( s^  n- 1)  is  non- increasing  in  s.  Therefore  (4)  is  non- increasing. 

Again  by  Proposition  4,  L /N  is  non- increasing  and  by  Lemna  2 which 

follows  V( 1^  n-1)  -( 1-p)  V(0,  n-1)  > 0.  Thus  (5)  is  also  non- increasing 

and  so  V( s,  n)  is  non- increasing  in  s > 1.  Since  V( s)  = lim  V(s^n), 

n -» 00 

V( s)  must  be  non- increasing  in  s > 1 also. 

The  proof  of  part  ( ii)  is  identical  except  for  the  use  of  part  (iii) 
of  Proposition  4 instead  of  part  ( i) . □ 

Lemma  1 . 

i)  V(0,n)  = 1 + &(l-r)  V(0,  n-1),  n > 0. 

ii)  V(0)  = 1 + s(i-r)  v(o). 

Proof.  These  results  follow  immediately  from  Proposition  3-  D 

Lemma  2. 

V(l,n)  > (1-p)  V(0,n)  , n > 0 . 

Proof.  The  proof  is  by  induction.  For  n = 0, 1 the  result  is  clearly 
true.  Now  assume  the  result  holds  for  n-1.  By  Proposition  5 

V(l,n)  > 1 + 5(K^(1-P)  V(0,n-1)/N^  + L^  V(  1,  n-l)/Np 

> 1 + &(K^(l-p)  V(0,  n-l)/N^  + L^(l-P)  V(0,  n.l)/N^) 

> 1 + &(1-P)  V{0,  n-1) 

> (1-p)  [1  + &(l-r)  V(0,  n-1)]  . 
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Thus  by  Lemma  1^ 


V(l,n)  > (1-0)  V(0,n) 


Theorem  2. 


i)  If  r < and  CCq  > then  V(0,n)  > V(  l,n)  and  V(0)  >V(1) 

ii)  If  r > then  V( 0,n)  < V( l,n)  and  V(0)  <V(1). 

Proof.  ( i)  For  n = 1 the  result  is  trivial.  Now  assume  V(O^n-l) 

> V(  1,  n-1).  Since  > a^0,  V(s,  n- 1)  is  non- increasing  in  s > 1 
by  Theorem  1^  so  via  Proposition  3 

V(l,n)  < 1 + &(K^(1.0)  + L^)  V(0,  n-l)/N^ 

< 1 + &(l-a^0)  V(0,  n-1) 

< 1 + &(l-r)  V(0,  n-1)  . 

Thus  by  Lemma  1^  V( l,n)  < V(0,n). 


(ii)  Clearly  V(0,n)  < V( 1,  n)  holds  for  n = 1.  Now  assume 
that  V(0  n-1)  < V( 1,  n-1).  By  Proposition  3 and  Lemma  1, 


V(l,n)  > 1 + 6(K^(l-p)  + V(0,  n- 1) 

> 1 + 6(  1 - a^p)  V(0,  n-1) 

> 1 + &( 1 - r)  V(0,  n-1)  = V(0,n)  , 

This  completes  the  proof  for  the  finite-horizon  optimal  values.  The 
results  regarding  the  infinite-horizon  optimal  values  follow  from 
equation  ( 1) . D 

In  the  case  omitted  by  Theorem  2^  a < and  it  can 

be  shown  that  neither  the  inequality  V(0,n)  >V(l,n)  nor  the  inequality 
V(O^n)  <V(l,n)  holds  in  general. 


1;.  Determination  of  Optimal  Inspection  Policies 

In  this  section  we  determine  the  form  of  the  optimal  inspection 


policies  for  various  ranges  of  the  parameters. 


Theorem  3.  i)  If  Ot^  > and  y < then  the  optimal  inspection 

policy  for  both  the  finite  and  infinite  horizon  problems  is  to  inspect 
every  period. 

(ii)  If  < a^P  and  y > Ctj^P,  then  the  optimal  inspection 
policy  for  both  the  finite  and  infinite  horizon  problems  is  to  never  inspect. 
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period  horizon  is  that  action  which  achieves  the  tnaxiinum  in  Proposition 
5.A.iii.  Let 


D(s,n)  = K^(l-P)  V(0,  n-l/N^  + V(  1,  n-l)/N^ 

- (K^(l-P)  + L^)  V(s+1,  n-l)/N^  . 

Then  when  the  device  is  in  state  s facing  a horizon  of  n periods  it 
is  optimal  to  inspect  if  D(s^n)  > 0 and  not  to  inspect  if  D(s,n)  < 0. 

When  CCq  > and  y < V(s^n)  is  non- increasing  in  s > 0 for  all 

n by  Theorems  1 and  2.  Thus  D(s,n)  > 0 for  all  s,n  > 1 and  it  is 
optimal  to  inspect.  When  and  T"  ^ V(s,n)  is  non-decreasing 

in  s > 0 for  all  n by  Theorems  1 and  2.  Thus  D(Sjn)  < 0 for  all 
s^n  > 1 and  the  optimal  policy  is  not  to  inspect.  This  completes  the 
finite  horizon  proof. 

The  optimal  action  when  the  device  is  in  state  s facing  an  infinite 
horizon  is  the  action  corresponding  to  the  argument  which  achieves  the 
maximum  in  Proposition  J.B.ii  (see  [9])«  To  be  more  specific^  let 

D(s)  = Kg(l-P)  V(0)/n^  + V(l)/Ng 

- (Kg(l-P)  + L^)  V(s+1)/N^  . 

Then  when  In  state  s it  is  optimal  to  inspect  if  D( s)  >0  and  not  to 
inspect  if  D( s)  < 0.  The  remainder  of  the  proof  is  identical  to  the 
finite  horizon  case.  □ 

The  lemma  which  follows  provides  the  basis  for  determining  the  form 
of  optimal  inspection  policy  when  and  y > 
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G(n)  = ap(l-P)  v(0,  n-1)  + ( 1-a^)  V(l,  n-1)  - ( l-a^t3)  V(2,n-1)  , 


n - 1^  2^  ...  j 


and 


H(s,n)  = D(s+1,  n-l)  - ( 1-Q^)  D(  s,  n-1) 


It  is  easy  to  verify  that  D(s^l)  = D( 3,2)  = 0 for  s = 1,  2,  ....  Also, 
G(  1)  =0  and  G(2)  = (C1!j^-Q!q)P  > 0.  Finally,  note  that  H(  s,2)  = 0 
[H(s,l)  is  undefined].  Thus  the  finite-horizon  portion  of  the  lemma 
holds  for  n = 1,  2.  Now  assume  the  ledma  holds  for  n = N-1,  where 
N > 5.  Let 

s*(n)  = min{s  > l:D(s,n)  > 0} 

where  s*(n)  = -h»  if  D(s,n)  <0  for  all  s.  Since  D(  s,  N-1)  is 
assumed  to  be  non-decreasing  in  s,  D(  s,  N-1)  is  non-negative  if  and 
only  it  s > s*(N-l).  Thus 
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(5) 


V(s,  N-1)  = L + 6(K^(1-0)  + Lg)  V(s+1,  N-2)/Ng  , 

for  s s*(N-l)  ^ 

and 

V(s,N-l)  = 1 + &K^(l-ti)  V(0,  N-2)/N^ 

+ &L^  V(l,  N-2)/Ng  , for  s > s*(N-l)  , 

In  the  remainder  of  the  proof  we  will  make  use  of  the  following  easily 

verified  formulas  for  K .L  . and  N without  mention: 

s*  s'  s 

'‘s,-!  * < Vs  . 

^.1'  <1-%)^  . (6) 

Case  1;  s*(N-l)  = 1:  In  this  case,  by  equations  (5)  and  Lemma  1 

D(s,N)  = {kJ1-P)  [1  + &(l-r)  V(0,  N-2)] 

+ LJl  + 6K^(1-P)  V(0,  N-2)/N^  + &L^  V(  1,  N-2)/N^] 

- (K^(l.p)  + L^)  [1  + V(0,  N-2)/N^^^ 

.SL^^lVd,  S-2)/B^^^]  { /N,  , 

= &(l-3)  (P-r)  v(0,  N-2) 


- [V(l,N-2)  - (l-P)  V(0,N-2)jLyN^  . 


(7) 


Recalling  Lenina  2^  Proposition  h and  the  assumptions  that  (i  > y and 
^1  ^ that  D(  s,N)  is  non-decreasing  in  s. 

Case  2:  s*(N-l)  > 1:  In  this  case  we  first  show  that  D( s^N)  is  non- 

decreasing in  the  ranges  [1^  s*(N-l)  - 1)  and  [s*(N-l)  - 1,  oo) . 


1 < s < s*(N-l)  - 1;  In  this  range  of  s^  by  equations  (5)  and  Lemma  1 

D(s,N)  = [1  + 6(l-y)  V(0,  N-2)]  ^ L J 1 + 6(K^(  1-P)+L^)  V(2,N-2)/N^] 

- ^ L^)  [1  + 6(K^^^(l-0)  + Lg^^)  V(8+2,  N-2)/N^^^]|/N^ 

= 6(l-r)(l-p)  V(0,N-2)Kg/N^  + 6(l-a^p)  V(2,  N-2)  L^/N^ 

- 6V(s+2,  N-2) 


6D(s+l,  N-1)  - &[(l-p)  V(0,  N-2)  + V(  1,N-2)L^^^/NJ 

+ 6(l-y)(l-0)  V(0,N-2)  K /N  + &(l-CLp)  V(2  N-2)  L /N 

s s s s 


&D(s+l,  N-1)N^^^/N^  + 6(l-P)0-r)  V(0,  N-2)  K^/N^ 


- BG(N-l)  L /N^  . 


(8) 


Since  N i/n  = 1 - 3K  /N  this  quantity  is  decreasing  when  a > Qt p 
S"t*  X s s s"  0 X 

and  y > CC^p.  Also^  D(  s+1^  N-1)  is  non-decreasing  by  the  Inductive 


assumption  and  in  the  range  s £ [ s*(N-l)  - 1)  it  is  negative.  Thus 
6D(s+l,  N-1)N^  non-decreasing.  Since  by  the  inductive  assumptic 
G(N-l)  > 0,  the  other  two  terms  in  the  above  expression  for  D(s^N)  are 
also  non-decreasing,  D(s,N)  is  non-decreasing  in  this  range  of  s. 


s*(N-l)  - 1 < s < oo:  In  this  range  of  s,  by  equations  (5)  and  Lemma  1 


D(s,N)  =|K^(l-t3)  [1  + &(l-r)  V(0,  N-2)] 

+ LJl  + 6(K^(l-P)  + L^  V(2,  N-2)/NJ 
- (K^(l-P)  4-  L^)  [1  + V(0,  N-2)/N^^^ 

= 8(l-r)  (l-p)  v(0,  N-2)Kg/N^  + 6(1  - V(2,  N-2)L^/N^ 

- &(l-p)  V(0,  N-2)Kg^^/N^  - &V(1,  N-2)L^^^/N^  , 


= 6(l-p)0-r)  V(0,  N-2)K  /N  - &G(N-1)L  /N  . 


Again  by  the  inductive  assumption  G(N-l)  > 0,  so  D( s,N)  is  non-decreasing 
in  this  range  of  s.  To  show  that  D( s,N)  is  non-decreasing  throughout 
[ l,oo)  all  that  remains  to  prove  is  that  D(s*(N-l)  - 1,  N)  - D(  s*(N- 1) -2,N)  > 0. 
Using  equations  (8)  and  (9)  and  denoting  s*(N-l)  for  the  time  being  simply 
by  3*, 
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D(s*-1,  N)  - D(s*-2,  N) 


= 6(l-ti)0-r)  V(0,  N-2)K^^_^/N^^_^  - &G(N-1)L^^_^/N^^_^ 

- 6D(s*-l,  N-1)N^^_^/N^^_2  - &(l-P)0-r)  V(0,  N-2)K^^_2/N^^_2 

, 6G(N-1)L^^_2/''s^-2 

= 5(l-P)0-r)  V(0,  N-2)  - K^^.2/Ns^.2] 

• 6G(N-1)  N-1)N^,./n^^_2  . 

Since  K /N  is  increasing  and  L /N  is  decreasing  the  first  two  terms 
s s s s 

in  the  above  expression  are  non-negative,  Also^  by  the  definition  of 
s*  = s*(N-l),  D(s*-1,  N-1)  < 0.  Thus  D(s*-1,  N)  - D(  s*-2,  N)  > 0.  Next 
we  show  that  H(s^N)  > 0 for  all  s. 

Case  1:  s*(N-2)  = 1:  Using  equation  (7)  and  simplifying, 

H(s,N)  = 6(l-p)(p.r)  V(0,  N-5)  - (l-aQ)K^J  > 0 

by  Lemma  4 (which  follows  this  proof). 

Case  2:  s*(N-2)  > 1,  s > s*(N-2)  - 1:  In  this  case,  using  equation  (9) 

and  simplifying, 

H(s,N)  = &(1-P)0.r)  V(0,  N-J)  - (i-ao)KJ  >0 

by  Lemma  4. 
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Case  j:  s*(N-2)  > 1,  s+1  = s>‘^(N-2)  - 1:  Using  equations  (8)  and  (9)  and 
simplifying 

H(s,N)  = b(l-pi)O-r)  v(0,  N-9)  - (l-aQ)KJ 

- &(1-Q!q)  D(s+1,  N-2)  . 

The  first  term  in  the  above  is  non-negative  by  Lemma  4.  The  second  term 
is  non-positive  because  s+1  < s*(N-2)  and  so  D(s+lj  N-2)  < 0. 

Case  4:  s+1  < s*(N-2)  - 1:  In  this  case,  using  equation  (8)  and  simplifying, 

H(s,N)  = 5(l-P)0-r)  V(0,  N-3)  ^ 6H(  s+1,  N- 1)  . 

Again  the  first  term  is  non-negative  bj  Lemna  4.  The  second  term  is 
non-negative  by  the  inductive  hypothesis.  Hence  we  have  shown  that 
H( s,N)  > 0 for  all  s. 

Next  we  show  that  G(N)  > 0. 

Case  1:  s*(N-l)  < 2:  By  Proposition  3.A.iii, 

V(l,  N-1)  > 1 + &Kj^(l-p)  V(0,  N-2)/N^  + 5L^  V(  1,  N-2)/N^  . 

Combining  this  inequality  with  the  expressions  for  V(0,  N-2)  and 
V(2,  N-2)  given  by  Leitma  1 and  equations  (5), 
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G(N)  >aQ(l-t3)  II  + b(l-r)  V(0,  N-2)l 

f (1%)  [1  + V(0,  N-2)  + &(l-a^)  V(1,  N-2)J  (10) 

- [1  + &K2(1-P)  V(0,  N-2)/N2  + bLg  V(  1,  N-2)/N2] 

> (a^-aQ)tj  + bv(o,  N-2)  [aQ(i-tJ)  ( i-r)  + a^(  i-p)(  i-a^)  - K^Ci-p)] 

+ 6V(  1,  N.2)  [ ( 1-a^)  ( i-a^)  - L^l 

> (Q;^-aQ)f3  + B(i-p)  - roiol  v(o,  N-2)  . ( ii) 

Since  > 0 and  p > y > 0,  G( N)  > 0. 

Case  2:  s*(N-l)  > 2.  In  this  case 

G(N)  = aQ(l-P)  [1  + &(l-r)  V(0,  N-2)] 

+ (l-a^)  [1  + b(K^(l-p)  + L^)  V(2,  N-2)/N^] 

- (I-O^p)  [1  + 5(K2(1-P)  + L2)  V(5,  N-2)/N2] 

= aQ(i-p)  [1  + &(i-r)  V(0,  N-2)] 

+ (i-a^)  [1  + &a^(i-H)  V(0,  N-2)  + B(i-a^)  v(  1,  n-2)] 

- (l-Oj^p)  [1  + &K2(l-ts)  V(0,  N-2)/N2  + &L2  v(l,  N-2)/N2l 

(l-a^)  [1  + 6N2  V(2,  N-2)  - 1 

- &a^(l-P)  V(0,  N-2)  - &(l-aj^)  V(l,  N-2)] 

- (1  - a^p)  [ I + bN^  V(3,  N-2)/N2  - 1 

- bK2(l-3)  V(0,  N-2)/N2  - bL2  V( 1,N-2)/N2]  . 

2h 


Now  the  first  three  terms  in  the  above  are  the  same  as  those  in  equation 
(10),  which  simplifies  to  (11).  Therefore 

G(N)  > (a^-aQ)(3  + 6(l-[3)  [a^p  - v(0,  N-2)  + &H(  1,N)  . 

Since  we  have  proved  above  that  H( 1,N)  > 0,  G(N)  > 0. 

It  remains  only  to  show  that  D( s)  is  non-decreasing.  By  equation 

(1) 

D( s)  = lim  D(s,n)  . 
n 00 

Since  each  D(s,n)  is  non- decreasing  in  s,  D( s)  must  be  non-decreasing 
also.  □ 

Lemma  4.  If  oc^  > ai3  then  K , > f l-a_)K  . 

0 s+1  — O''  s 

Proof.  Case  1:  > P:  Here 

■'s.r  • 

“o  < ■ “o”s  ■ • \I“o  ■ 

sign  of  the  above  expression  is  determined  by  that  of  - BK  /N  and 

0 s s 

since  this  quantity  is  decreasing  all  we  have  to  show  is  that 

U.  KyN^<C<^P  . 

S -»  00 

From  Proposition  2, 
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s-  ] 


K /N 
s s 


1) 


(v«  * “o<‘-“i'  ■ *'  » <‘-“i>  <Vrt 


Thus 


u„  K^/N^  » ayp  . 

S -»  00 

Theorem  U.  If  *^q  ^ T ^ then 

i)  if  the  device  is  in  state  s facing  an  n-period  horizon^  it  is 
optimal  to  inspect  if  and  only  if  s > s*(  n) , 
ii)  if  the  device  is  in  state  s facing  an  infinite  horizon^  it  is 
optimal  to  inspect  if  and  only  if  s > s*(oo)  min{s:D(s)  > Oj. 

Proof.  This  theorem  is  an  immediate  consequence  of  Lemma  3. 


The  form  of  the  optimal  inspect  ion  policy  has  been  analyzed  in 
every  case  except  o;^  and  y < in  this  case  it  turns  out 

that  D(s,n)  crosses  zero  at  most  once  and  from  above  (s  varying, 
n fixed)  and  so  it  is  optimal  to  inspect  in  state  s unless  s is 
larger  than  some  number  z*(n). 

Lemma  ^ . If  C(^  < and  y < then  as  s increases  D(s,n)  and 

D( s)  cross  zero  at  most  once  and  from  above. 


Proof. 

D(s,n)  = [V(0,n-1)  - V(s+1,  n- 1)  ] ( l-P)Kg/N^ 

+ [V(l,  n-1)  - V(s+1,  n-l)]L/N  . (12) 

By  Theorem  l.ii,  V(s+1,  n-1)  is  non-decreasing  in  s and  by  Proposition 

4.iii  is  decreasing  and  is  increasing.  Consider  the 

interval  of  s for  which  V(0,  n-1)  > V(s+1,  n-1).  In  this  range 

V(0,  n-1)  - V(s+1,  n-1)  is  non- negative  and  is  non- increasing.  Thus  in 

this  range  the  first  term  in  (12)  is  non- increasing.  Also  V(  1,  n-1)  - V(s-hI) 

is  non- positive  and  non- increasing,  so  the  second  term  is  also  non- increasing. 

Thus  D(s,n)  is  non- increasing  for  s in  this  range.  For  the  interval  of 

s for  which  V(0,  n-1)  < V(s+1,  n-1),  both  terms  in  (12)  are  negative  and 

thus  D(  s,  n-1)  < 0.  Thus  D(s,n)  crosses  zero  at  most  once  and  from 

above,  as  desired.  Since  D( s)  = lim  D(s,n)  it  is  easy  to  verify  that 

n — ♦ 00 

D(  s)  must  also  have  this  property.  U 


z*(n)  = niax{s:D(  Sjti)  > 0} 


and 

z*(oo)  = tnax{s:D(s)  > 0) 

where  we  cake  the  maximum  over  an  empty  set  to  be  -oo. 

Theorem  ^ . If  0!^  < and  y < then  if  the  device  is  in  state  s 

facing  an  n-period  horizon^  it  is  optimal  to  inspect  if  and  only  if 
s < z*(  n)  j ( 1 < n < oo) . 

Proof:  Follows  immediately  from  Lemma  5- 


5 . Conclusions 

The  form  of  the  optimal  policy  has  been  shown  to  depend  in  a 

simple  way  upon  y,  and  The  quantities  and  y have 

obvious  interpretations^  is  the  probability  that  given  the  device  is 

currently  in  observed  state  I (i.e.,  last  inspection  one  period  ago; 

results  OK)  that  the  device  will  be  failed  at  the  start  of  the  next  period. 

When  y < observed  state  0 seems  intuitively  to  be  a better  state  to 

be  in  than  state  1,  and  conversely.  The  relation  between  and 

has  been  shown  to  determine  whether  or  not  the  conditional  probabilities 

K /n  amd  L /N  are  increasing  or  decreasing.  With  these  facts  noted^ 
s s s s 

the  forms  of  the  optimal  policy  in  the  various  cases  are  intuitively  sensible. 


The  optimal  policy  when  Cl^  < y < Ot^fi  perhaps  deserves 

some  special  discussion  because  of  its  seemingly  odd  nature.  When  the 
device  is  in  observed  state  s^  inspection  may  seem  intuitively 

desirable  since  if  the  true  state  is  2^  then  in  every  period  in 
which  the  partial  failure  remains  undetected  there  is  a larger  probability 
of  complete  failure.  When  y > this  preference  for  inspection  indeed 

should  hold  for  s = 1.  However^  when  ^s^^s  increasing, 

so  that  as  s increases  the  probability  that  the  device  is  in  true  state 
1 (OK)  increases  also.  Of  course  when  the  true  state  is  1,  inspection 
is  not  desirable.  Thus  it  seems  reasonabi;^  that  once  the  observed  state  s 
is  larger  than  some  number,  ^g/^g  sufficiently  large  to  make  not 
inspecting  optimal.  Finally  we  should  note  that  when  and 

y < the  optimal  policy  for  a device  starting  out  in  observed  state 

s < z*(oo)  will  be  to  always  inspect,  and  so  the  device  will  never  in 
the  future  reach  any  observed  state  greater  than  1.  However  if  the  device 
starts  out  in  observed  state  s > z*(a)),  then  the  optimal  policy  will  be  to 


never  inspect. 


r 
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impaired  may  itself  cause  the  device  to  become  impared.  In  this  respect 
inspection  is  hazardous.  This  paper  deals  with  the  determination  of  inspec- 
tion  policies  which  maximize  the  expected  lifetime  of  the  device.  
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